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OCT  -  S  20(18 

MEMORANDUM  FOR  AUDITOR  GENERAL.  DEPARTMENT  OF  THE  NAVY 

SUBJECT:  Quality  Control  Review  of  Naval  Audit  Service’s  Special  Access  Program 
Audits  (Report  No.  D-2009-6-001) 

We  are  providing  this  report  for  your  information  and  use.  We  reviewed  the 
Naval  Audit  Service’s  (N  AS)  system  of  quality  control  over  Special  Access  Programs 
(SAP)  audits  for  the  two  years  ended  September  30,  2007.  The  Government  Auditing 
Standards  (GAS)  requires  that  an  audit  organization  performing  audits  and/or  attestation 
engagements  in  accordance  with  GAS  should  have  an  appropriate  internal  quality  control 
system  in  place  and  undergo  an  external  peer  review  at  least  once  every  three  years  by 
reviewers  independent  of  the  audit  organization  being  reviewed.  As  the  organization  that 
has  audit  policy  and  oversight  responsibilities  for  audits  in  the  DoD,  we  conducted  the 
external  quality  control  review  of  the  NAS's  SAP  audits  in  conjunction  with  the  Army 
Audit  Agency’s  (AAA)  review  of  the  NAS  non-SAP  audits. 

An  audit  organization’s  quality  control  policies  and  procedures  should  be 
appropriately  comprehensive  and  suitably  designed  to  provide  reasonable  assurance  of 
meeting  the  objectives  of  quality  control.  We  tested  the  NAS  SAP  system  of  quality 
control  for  audits  to  the  extent  considered  appropriate. 


In  our  opinion,  the  system  of  quality  control  for  the  audit  function  of  NAS  SAP  in 
effect  for  the  period  ended  September  30.  2007  was  designed  in  accordance  with  quality 
standards  established  by  GAS.  Further,  the  internal  quality  control  system  was  operating 
effectively  to  provide  reasonable  assurance  that  SAP  audit  personnel  were  following 
established  policies,  procedures,  and  applicable  auditing  standards.  Accordingly,  we  are 
issuing  an  unmodified  opinion  on  your  SAP  audit  quality  control  system  for  the  review 
period  ended  September  30,  2007. 


Appendix  A  contains  comments,  observations,  and  recommendations  where  NAS 
can  improve  its  quality  control  system.  Appendix  B  contains  scope  and  methodology  of 
the  review  and  Appendix  C  provides  the  full  text  of  management  comments  in  response 
to  the  draft  report.  Questions  should  be  directed  to  Mr.  Robert  Kienitz  at  (703)  604-8754 
(DSN  664-8754),  Robert . K  ieni tz@d od i g . m i  1 . 

Carolyn  R.  Davis 
Assistant  Inspector  General 
for  Audit  Policy  and  Oversight 


Appendix  A.  Comments,  Observations,  and 

Recommendations 


We  are  issuing  an  unmodified  opinion  because  we  determined  that  the  NAS  quality  control 
system  is  adequately  designed  and  functioning  as  prescribed.  The  concerns  we  identified  with 
the  findings,  conclusions,  or  recommendations  during  our  review  of  the  selected  NAS  audit 
reports  were  not  cumulatively  significant  enough  to  indicate  that  material  deficiencies  existed  in 
the  NAS  quality  control  system  for  complying  with  GAS.  Because  of  the  timeframe  of  the  audit 
reports  in  our  review,  we  measured  the  audits  for  compliance  with  the  2003  revision  of  the  GAS 
and  the  July  2006  NAS  Audit  Handbook.  We  identified  issues  that  are  still  applicable  even 
when  we  apply  the  updated  2007  revision  of  GAS  and  the  May  2008  NAS  Audit  Handbook. 

Although  the  concerns  we  identified  did  not  affect  our  overall  opinion,  there  were  areas  where 
NAS  could  improve  the  quality  control  process.  We  originally  judgmentally  selected  three  audit 
reports  issued  during  a  two  year  time  period  ending  September  30,  2007;  however,  we  were  only 
able  to  review  two  of  the  audit  reports  because  of  additional  security  requirements  for  one  of  the 
reports  in  the  original  sample.  We  tested  the  reports  for  compliance  with  GAS  and  NAS  audit 
policies  in  nine  areas  to  include  independence,  professional  judgment,  competence,  audit 
planning,  supervision,  evidence  and  audit  documentation,  reporting,  non-audit  services,  and 
quality  control.  We  identified  minor  discrepancies  in  three  of  the  nine  areas  in  our  review  in 
applying  NAS  audit  policy  relating  to: 

•  audit  planning, 

•  evidence  and  documentation,  and 

•  quality  control. 

Audit  Planning.  GAS  7.41  (2003  Revision)  states  that  a  written  audit  plan  should  be  prepared 
for  each  audit.  The  fonn  and  content  of  the  written  audit  plan  will  vary  among  audits  but  should 
include  an  audit  program  or  project  plan,  a  memorandum,  or  other  appropriate  documentation  of 
key  decisions  about  the  audit  objectives,  scope,  and  methodology  and  of  the  auditors’  basis  for 
those  decisions.  It  should  be  updated  as  necessary,  to  reflect  any  significant  changes  to  the  plan 
made  during  the  audit.  In  addition,  GAS  7.42  states  that  documenting  the  audit  plan  is  an 
opportunity  for  the  auditors  to  supervise  audit  planning. 

NAS  Audit  Handbook  (July  2006,  Section  406)  requires  that  a  survey  plan/program  will  be 
developed  early  in  the  audit  to  gather  the  infonnation  that  will,  in  turn,  comprise  the  content  of 
the  fonnal  survey  debrief.  NAS  Audit  Handbook  (July  2006,  Section  415,  paragraph  2)  requires 
that  the  responsible  Associate  Director  must  approve  the  audit  program  before  the  start  of  the 
audit  verification  phase. 
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We  found  that  NAS  did  not  adequately  plan  for  one  of  the  two  audits  in  our  review.  One  audit  in 
the  review  was  an  audit  assist,  and  an  audit  plan  had  not  been  prepared.  The  audit  fdes  for  this 
report  also  did  not  include  documentation  to  indicate  why  an  audit  plan  had  not  been  prepared. 
For  another  audit  in  our  review,  a  survey  program  had  been  prepared,  but  we  could  not  detennine 
the  date  that  the  survey  program  had  been  prepared.  The  steps  for  the  survey  were  included  as 
part  of  the  audit  program.  While  there  were  no  dates  to  indicate  when  the  auditors  prepared  the 
survey  program,  there  was  documentation  in  the  fdes  to  show  that  survey  steps  were  completed, 
and  the  auditors  presented  their  results  of  the  survey  phase  in  a  ninety  day  survey  debrief  to  NAS 
management. 

Recommendation:  We  recommend  that  the  Auditor  General  of  the  Navy  issue  a 
memorandum  to  remind  all  SAP  audit  personnel  of  the  importance  of  complying  with 
established  guidance  for  preparing  survey  and  audit  programs  early  in  the  audit. 

Management  Comments.  The  Auditor  General  concurred  with  the  recommendation  and 
stated  that  an  e-mail  will  be  sent  to  the  audit  staff  to  remind  them  of  the  importance  of  complying 
with  the  established  guidance  for  preparing  survey  and  audit  programs  early  in  the  audit. 

Reviewer  Response.  Management  comments  are  responsive. 

Evidence  and  Documentation.  GAS  7.66  (2003  Revision)  states  that  auditors  should  prepare 
and  maintain  audit  documentation.  GAS  7.66  also  states  that  audit  documentation  should 
contain  support  for  findings,  conclusions,  and  recommendations  before  auditors  issue  their 
report. 

NAS  Audit  Handbook  (July  2006,  Section  501)  reiterates  the  GAS  by  stating  that  auditors  should 
prepare  and  maintain  audit  documentation,  and  the  audit  documentation  should  contain  support 
for  findings,  conclusions,  and  recommendations  before  auditors  issue  their  report. 

We  judgmentally  selected  ten  facts  and/or  figures  from  the  working  papers  for  one  of  the  audit 
reports  in  our  review.  We  found  that  all  of  the  ten  facts  and/or  figures  were  supported  and 
included  working  paper  elements  such  as  purpose,  source,  scope,  and  conclusion  as  required  in 
the  NAS  Audit  Handbook.  For  the  other  audit  report  in  our  review,  we  judgmentally  selected 
fifteen  facts  and/or  figures  in  the  audit  report.  We  reviewed  the  supporting  working  papers  for 
evidence  and  documentation  and  supervisory  reviews.  We  found  that  one  of  the  figures  had  not 
been  properly  updated  in  the  working  paper  after  the  independent  reference  review.  While  the 
auditor  did  not  properly  update  the  working  paper  to  reflect  the  change,  the  updated  figure  was 
corrected  in  the  audit  report.  Though  the  information  was  not  properly  documented,  it  did  not 
have  a  significant  impact  on  the  results  or  the  conclusions  presented  in  the  report. 

In  addition,  we  were  unable  to  locate  a  working  paper  in  the  audit  files  that  supported  at  least 
two  facts/figures  in  the  audit  report.  The  facts  relating  to  the  working  paper  did  not  have  a 
significant  impact  because  they  did  not  affect  the  results  and  conclusions  addressed  in  the  report. 
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Recommendation:  We  recommend  that  the  Auditor  General  of  the  Navy  issue  a 
memorandum  to  remind  all  SAP  audit  personnel  of  the  importance  of  complying  with 
established  guidelines  for  documenting  evidence  to  include  maintaining  information  in  the  audit 
fdes  and  updating  audit  work  included  in  the  audit  reports. 

Management  Comments.  The  Auditor  General  concurred  with  the  recommendation  and 
stated  that  an  email  will  be  sent  to  the  audit  staff  to  remind  them  of  the  importance  of  complying 
with  established  guidelines  for  documenting  evidence  to  include  maintaining  information  in  the 
audit  fdes  and  updating  audit  work  included  in  the  audit  reports. 

Reviewer  Response.  Management  comments  are  responsive. 

Quality  Control.  GAS  3.49  (2003  Revision)  requires  that  each  audit  organization  performing 
audits  and/or  attestation  engagements  in  accordance  with  GAS  should  have  an  appropriate 
internal  quality  control  system  in  place.  Also,  GAS  3.51  states  that  each  audit  organization 
should  prepare  appropriate  documentation  for  its  system  of  quality  control  to  demonstrate 
compliance  with  its  policies  and  procedures. 

NAS  Audit  Handbook  (July  2006,  Section  857)  states  that  editors  perform  a  mandatory  check  to 
ensure  that  draft  reports  conform  to  the  reporting  standards  for  content  and  style.  The  editor 
provides  copies  of  a  completed  guide  sheet  to  the  applicable  Program  Manager,  Associate 
Director,  and  Assistant  Auditor  General  who  are  responsible  for  making  the  appropriate 
corrections.  The  Program  Manager  and/or  Associate  Director  should  ensure  the  completed  guide 
sheets  are  included  in  the  audit  files. 

The  NAS  used  quality  control  procedures  such  as  supervisory  initials  and  dates  on  working 
papers,  and  independent  reference  reviews  for  draft  and  final  audit  reports.  One  of  the  audits  in 
our  review  was  an  audit  assist  which  used  a  letter  report  format.  The  audit  files  did  not  contain  a 
checklist  identifying  the  editor’s  review.  The  NAS  had  not  developed  an  editor’s  guide  sheet  for 
letter  reports.  The  AAA  recommended  in  their  FY  2008  peer  review  of  the  NAS  non-SAP  audits 
for  the  NAS  to  develop  an  editor’s  guide  sheet  for  letter  reports.  The  NAS  management 
commented  that  they  would  develop  an  editor’s  guide  sheet  for  letter  reports,  attestation  reports, 
and  opinion  reports.  In  response  to  the  AAA  recommendation,  the  NAS  editor’s  have  developed 
a  draft  of  the  editor’s  guide  sheet  for  letter  reports.  They  are  currently  in  the  process  of  refining 
the  draft  guide  sheet  for  use  by  editors  for  letter  reports,  attestation  reports,  and  opinion  reports. 
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Appendix  B.  Scope  and  Methodology 


We  limited  our  review  to  the  adequacy  of  NAS  SAP  audits’  compliance  with  quality  policies, 
procedures,  and  standards.  We  originally  judgmentally  selected  three  SAP  audits  from  a 
universe  of  formal  reports  issued  by  NAS  SAP  auditors  during  FY  2006  and  FY  2007,  but 
limited  our  review  to  two  audits  because  of  additional  security  requirements  for  access  to  the 
third  report.  We  tested  each  audit  for  compliance  with  the  NAS  system  of  quality  control.  The 
AAA  conducted  a  review  of  the  NAS  internal  quality  control  system  for  non-SAP  audits  and/or 
attestation  engagements  and  has  issued  a  separate  report.  The  Acting  Assistant  Inspector 
General  for  Audit  Policy  and  Oversight  will  issue  an  overall  opinion  report  on  the  NAS  internal 
quality  control  system  that  will  include  the  combined  results  of  the  SAP  and  non-SAP  audit 
reviews. 

In  performing  our  review,  we  considered  the  requirements  of  quality  control  standards  and  other 
auditing  standards  contained  in  the  2003  Revision  of  the  GAS  issued  by  the  Comptroller  General 
of  the  United  States.  GAS  3.52  states: 

The  external  peer  review  should  determine  whether,  during  the  period  under  review,  the  reviewed  audit 
organization’s  internal  quality  control  system  was  adequate  and  whether  quality  control  policies  and 
procedures  were  being  complied  with  to  provide  the  audit  organization  with  reasonable  assurance  of 
conforming  with  applicable  professional  standards.  Audit  organizations  should  take  remedial,  corrective 
actions  based  on  the  results  of  the  peer  review. 

We  conducted  this  review  in  accordance  with  standards  and  guidelines  established  in  the  April 
2005  President’s  Council  on  Integrity  and  Efficiency  (PCIE)  “ Guide  for  Conducting  External 
Peer  Reviews  of  the  Audit  Operations  of  Offices  of  Inspector  General.'’'’  We  used  a  modified 
guide  to  ensure  consistency  with  the  AAA  review  of  non-SAP  audits,  and  to  reflect  the  unique 
nature  of  auditing  within  a  SAP  environment.  We  reviewed  audit  documentation,  interviewed 
NAS  auditors,  and  reviewed  NAS  internal  audit  policies.  We  reviewed  the  DoD  OIG 
Report  No.  D-2005-6-010,  “Quality  Control  Review  of  the  Naval  Audit  Service’s 
Special  Access  Program  Audits”  dated  September  2,  2005,  and  the  AAA’s  Letter  of  Comments 
on  the  Fiscal  Year  2008  External  Quality  Control  Peer  Review  of  the  Naval  Audit  Service, 
Report:  A-2008-01 15-  PMZ  dated  June  12,  2008.  We  perfonned  this  review  from 
July  to  September  2008  at  one  NAS  field  office. 

We  used  the  following  criteria  to  select  the  audits  under  review: 

•  Worked  backward  starting  with  the  FY  2007  audits  in  order  to  review  the  most  current 
quality  assurance  procedures  in  place. 

•  Avoided  audits  with  multiple  SAPs  associated  with  the  audits  for  ease  of  access. 

•  Avoided  audits  that  have  the  same  or  similar  titles,  to  ensure  review  of  multiple  types  of 
projects. 
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The  following  table  identifies  the  specific  reports  reviewed. 


Report  Number 

Date 

Title 

N-2006-0046 

September  21,  2006 

National  Security  Agency  Military 
Interdepartmental  Purchase  Request 

N-2007-0049 

January  30,  2007 

Intelligence  Related  Contracts  (IRC) 

Limitations  of  Review.  Our  review  would  not  necessarily  disclose  all  weaknesses  in  the  system 
of  quality  control  or  all  instances  of  noncompliance  with  it  because  we  based  our  review  on 
selective  tests.  There  are  inherent  limitations  in  considering  the  potential  effectiveness  of  any 
quality  control  system.  In  performing  most  control  procedures,  departures  can  result  from 
misunderstanding  of  instructions,  mistakes  of  judgment,  carelessness,  or  other  human  factors. 
Projecting  any  evaluation  of  a  quality  control  system  into  the  future  is  subject  to  the  risk  that  one 
or  more  procedures  may  become  inadequate  because  conditions  may  change  or  the  degree  of 
compliance  with  procedures  may  deteriorate. 
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Appendix  C.  Naval  Audit  Service  Comments 


DEPARTMENT  OF  THE  NAVY 

NAVAL  AUDIT  SERVICE 
1006  BEATTY  PLACE  SE 
WASHINGTON  NAVY  YARD.  DC  20374-5005 


7510 
2  Oct  08 

MEMORANDUM  FOR  DEPARTMENT  OF  DEFENSE  INSPECTOR  GENERAL 

(ASSISTANT  INSPECTOR  GENERAL  FOR  AUDIT  POLICY 
AND  OVERSIGHT) 

Subj:  RESPONSE  TO  DEPARTMENT  OF  DEFENSE  INSPECTOR  GENERAL  DRAFT 
QUALITY  CONTROL  REVIEW  REPORT  OF  NAVAL  AUDIT  SERVICE’S 
SPECIAL  ACCESS  PROGRAM  AUDITS 

Ref:  (a)  Memorandum  for  Auditor  General.  Department  of  the  Navy  -  Quality  Control 
Review  of  Naval  Audit  Service's  Special  Access  Program  Audits 
(Project  No.  D2008-DIPOAI-0090.000) 

Enel:  ( 1 )  Department  of  Defense  Inspector  General  (Assistant  Inspector  General  for  Audit 
Policy  and  Oversight)  Recommendations  and  Naval  Audit  Sen  ice  Responses 

1 .  We  have  reviewed  your  report  on  our  Special  Access  Program  audits  (reference  (a)),  and  are 
pleased  that  Department  of  Defense  Inspector  General  has  provided  us  with  an  unmodified 
opinion  on  our  system  of  quality  control  for  the  year  ended  30  September  2007.  Enclosure  (1 ) 
contains  our  response  to  the  recommendations  made  in  the  report. 

2.  If  you  have  any  questions,  or  would  like  additional  information,  please  contact  Ms.  Vicki 
McAdams.  Director  Policy  and  Oversight,  at  Vicki. McAdams/i  navv.mil  or  (202)  433-5854. 


SAMUEL  CHASON 

Assistant  Auditor  General 

Plans.  Policy  and  Resource  Management 
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Department  of  Defense  Inspector  General  (Assistant 
Inspector  General  for  Audit  Policy  and  Oversight) 
Recommendations  and  Naval  Audit  Service  Responses 

Overall  Comment 

We  are  pleased  to  receive  an  unmodified  opinion  on  our  quality  control  system 
for  Special  Access  Program  (SAP)  audits.  We  have  evaluated  the  report  and 
concur  with  the  recommendations. 

Department  of  Defense  Inspector  General 
Recommendations  and  Naval  Audit  Service  Responses 

Recommendation  1.  We  recommend  that  the  Auditor  General  issue  a 
memorandum  to  remind  all  SAP  audit  personnel  of  the  importance  of  complying 
with  established  guidance  for  preparing  survey  and  audit  programs  early  in  the 
audit. 

Naval  Audit  Service  Response.  Concur.  An  All-Hands  e-mail  will  be  sent  to 
the  staff  reminding  them  of  the  importance  of  complying  with  established 
guidance  for  preparing  survey  and  audit  programs  early  in  the  audit.  Action 
will  be  complete  by  10  October  2008. 

Recommendation  2.  Wc  recommend  that  the  Auditor  General  issue  a 
memorandum  to  remind  all  SAP  audit  personnel  of  the  importance  of  complying 
with  established  guidelines  for  documenting  evidence  to  include  maintaining 
information  in  the  audit  files  and  updating  audit  work  included  in  the  audit  reports. 

Naval  Audit  Service  Response.  Concur.  An  All-1  lands  e-mail  will  be  sent  to 
the  staff  reminding  them  of  the  importance  of  complying  with  established 
guidelines  for  documenting  evidence,  to  include  maintaining  information  in  the 
audit  files  and  updating  audit  work  included  in  the  audit  reports.  Action  will 
be  complete  by  10  October  2008. 
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